SYSTEM AND METHOD FOR LIVENESS AUTHENTICATION USING AN 
AUGMENTED CHALLENGE/RESPONSE SCHEME 

FIELD OF THE INVENTION 

This invention relates to the field of image processing. More specifically, this invention relates 
5 to authenticating acquired images using a sensor. More specifically, the invention deals with 
authenticating the originality of biometrics data like fingerprint images. 

BACKGROUND OF THE INVENTION 

With the rapid growth of the Internet, many new e-commerce applications are being developed, 
and deployed. For example, retail purchasing and travel reservations over the web using a credit 

10 card are very common commercial applications. Today, the users are recognized using a userlD 
and password combination for identification and authentication purposes. Very soon, more 
secure methods for authentication and possibly identification involving biometrics, such as 
fingerprint and face images, will be replacing these simple methods of identification. An 
automated biometrics system involves acquisition of a signal from the user that more or less 

15 imiquely identifies the user. For example, in fingerprint-based authentication a user's fingerprint 
needs to be scanned, in a speaker verification system a user's speech signals is recorded. 

In many unattended apphcations, the server or other computing device has the burden of ensuring 
that the data transmitted firom a cUent is current and live information, not previously acquired or 
otherwise constructed or obtained information. With the rapid grovrth of the Intemet, users are 
20 expected to be involved in a variety of remote unattended transaction applications. The 

application server has to ensure that the transmitted information is firesh and has been acquired at 
the time requested, even more so in the case when any multimedia information is being 
submitted. It is very easy to store digital multimedia information and recycle the stale 
information to the server. In many e-commerce applications, if the multimedia information is 
being used in a time-sensitive fashion, it is very important to ensure the authenticity of the 
multimedia information for successful operation of the system. 

One of the main advantages of Internet-based business solutions is that they are accessible fi'om 
remote, unattended locations including the user's home. However, the biometrics signals can be 
intercepted or stored on the local machine in these remote and xmattended locations or otherwise 
obtained firom applications where the user uses her/his biometrics. The recorded signals can then 
later be reused for unknown, fi-audulent purposes such as to impersonate a registered user of an 
Intemet service. The simplest method is that a signal is acquired once and reused several times. 
Simple perturbations can be added to this previously acquired signal to give an impression that it 
is fi-esh. Detection of such misuse is difficult to determine at the server side. A more 
sophisticated attacker might create phony Intemet businesses and acquire a copy of a user's 
biometric signal and then with intelligent modifications to the signal, pose as this user to other 
Intemet service providers. The financial implications of such attacks can be substantial. 
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Fingerprints have been used for identifying persons for several decades. In an automatic 
fingerprint identification system, the first stage is the image acquisition stage where the subject's 
fingerprint is sensed. The acquired image is then processed and matched against a stored 
template. There are several techniques to acquire a fingerprint including scanning an inked 
5 fingerprint, and inkless techniques using optical, capacitative and other semiconductor-based 
sensing techniques. These sensing techniques typically locate ridges and valleys in the 
fingerprint. 

PROBLEMS WITH THE PRIOR ART 

There are ways to protect multimedia data and other information during transmission using 
cryptography techniques. Some well- known cryptographic techniques include RSA, DES, PKI 
and digital signatures. The crypto techniques running on the client can still be fed previously 
acquired images or multimedia data as there is no secure link between the near end of the 
encrypted communication channel and the image sensor. One way to verify the authenticity of an 
image is to compute a signature based on that image and transmit it in addition to the image. 
However, provided the encryption remains secure, all this ensures is that the image was acquired 
by a valid device. It does not say when it was acquired. 

A challenge/response system poses a challenge to the client and verifies the response against the 
enrolled response. Examples include asking a question like the user's mother's maiden name as 
the challenge and verifying that the same answer was provided as during enrollment. 
20 Challenge/response systems can also be constructed where a secret software algorithm is used to 
automatically generate a proper response to a digital challenge string. The standard 
challenge/response approach used in many on-line transaction systems keeps the image sensor 
out of the loop. There is still an open path for attack between the image sensor and the processor. 
That is, one can feed images directly into the processor, bypassing the sensor. 

25 Using encryption methods to securely transmit the biometrics signal deters snooping the network 
and makes decrypting intercepted transmissions virtually impossible. But locally on the client 
computer these techniques have no impact. 

Today the biometrics sensors such as a camera, fingerprint scarmer or a microphone do not have 
bi-directional transmission and do not have the computational capacity to compute any sort of 
30 digital signature on the sensor. Yet, any hardware separation between the real sensor and the 
client computer leaves security holes in the system. In summary, while the prior art can access 
and securely transmit biometric information/images (fingerprints) ~ prior art fails to verify that 
the biometrics information was obtained fi"om the proper source at the proper time. 

Friedman (U.S. Patent number 5,499,294) describes a digital camera with a processor for 
35 authentication of images. The camera stores a private and public key pair to compute a hash 
function of the image. The hash function can be verified by using the public key. There are no 
external challenges involved in this system. There is no attempt to guarantee liveness. Also, the 
adopted mode of cryptography is known to be compute intensive and involve management of the 
keys. 
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Haber and Stometta (U.S. Patent number 5,136,647) describe a method for secxire time stamping 
a digital document including images and audio signals. The method validates only their temporal 
existence by using a trusted server-based time stamping scheme. This time stamping is done 
remotely, not at the source of the signal. The method does not guarantee liveliness of the signal, 
5 i.e., if the same signal is sent to system twice with a time interval, the system would validate both 
of them. 

Bennet et al (U.S. Patent number 5,764, 769) describe a method to authenticate digital video and 
audio recordings. In their method the authenticating video camera uses the challenge to set the 
tunable parameters. The trusted repository stores the challenges and the snapshot of the video. In 
10 a biometrics system often the best images are obtained by dynamically adjusting the camera or 
sensor parameters to acquire the best possible image. Hence, setting camera parameters, as would 
be required by Bennet, is not advisable. Secondly, the system involves a storage server which is 
not required in a transaction-based system. 

These above mentioned patents and the following references give further backgroxmd and are 
1 5 incorporated by reference in their entirety: 

Stuart A. Haber and Wakefield S. Stometta, Jr., 
Method for secure time-stamping of digital documents 
U.S. Patent number 5,136,647, Aug. 1992 

Gary L. Friedman, 

20 Digital camera with apparatus for authentication of images produced firom an image file 
U.S. Patent number 5,499,294, Mar. 1996 

Charles Henry Bennet, David Peter DiVincenzo and Ralph Linsker, 

Digital recording system with time-bracketted authentication by on-line challenges and method 
of authenticating recordings 
25 U.S. Patent number 5,764,769, June 1998 

OBJECTS OF THE INVENTION 

An object of this invention is an improved system and method for authenticating multimedia 
information signals like images and video. 

An object of this invention is an improved system and method for authenticating biometric 
30 information like fingerprint images, face images/video, speech signals. 

An object of this invention is an improved system and method for ensuring that the biometric 
provided/transmitted/sensed is a newly acquired biometric. 
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An object of this invention is an improved system and method for ensuring that the biometric 
provided/transmitted/sensed is a biometric newly acquired by a specific sensor. ' 



SUMMARY OF THE INVENTION 

The present invention achieves these and other objectives by ensuring that a computer (e.g. a 
server) authenticates the signal source obtained (e.g. at the client). A challenge/response system 
is augmented by data from the sensor acquiring the biometric (e.g. fingerprint) and a| response is 
computed on the same physical sensor so that there is a significantly reduced opportunity for 
tampering. Portions of the augmentation parameters can be easily modified to address the 
situation where the current algorithm used for computing the response has been uncovered by 
criminally intended computer hackers. 

The system has one or more acquisition devices for creating a representation of one or more input 
signals. A signal output is connected to one or more application devices. A responder receives 
the signal representations from the acquisition devices, in response to challenges received from 
one or more challenge generator devices, and the responder sends responses that are a function 
of the signals and the challenges. These signal representations can be verified by comparing the 
responses to the function of the signals and the challenges. 

These basic techniques can be extended to acquisition devices generating signal representation 
from various types of multimedia such as voice and video signals. 

BRIEF DESCRIPTION OF THE DRAWING 

Figure 1 is a block diagram of one preferred embodiment of the present invention in a networked 
application. 

Figure 2 is a block diagram of one of the clients. 

Figure 3 is a block diagram of a hardware embodiment of a responder in a client. 

Figure 4 is a flow chart of a challenge responder embodied as a process in the sensor at the client. 

Figure 5 is a block diagram of a preferred server. 

Figure 6 is a flow chart a transaction process as performed by the server. 

DETAILED DESCRIPTION OF THE INVENTION 
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This invention relates to authenticating acquired images using a sensor. More specifically, it 
deals with authenticating the originahty of biometrics data like fingerprint images. By adding a 
signature specifically computed from the acquired image in the sensor, authenticity of the image 
is ascertained. The integrated sensor has the power to augment an external challenge using the 
5 image and compute the response to the augmented challenge. Previously acquired or otherwise 
constructed or obtained digital images being fraudulently resubmitted to a server can be rejected 
using this technique. 

The invention has components on the sensor and other components on the server. The 
components on the sensor are: (i) a challenge augmentor; and (ii)a response generator. On the 
10 server, the components are: (i) a challenge generator, (ii)a response verifier, and (iii)a list of 
which response algorithm is being used by each client. 

In general, a challenge is an arbitrary string of bits. It is desirable that the challenges repeat very 
seldom, if ever. It must also be difficult to predict the next challenge from a history of previous 
challenges or commonly available information such as the time of day. The response generator 

15 likewise produces a string of bits which depends on the particular challenge issued. However, in 
a preferred embodiment, the same challenge should not always generate the same response. It is 
best if the response generator also considers the properties of the image as well in generating its 
answer. In effect, the original challenge has been augmented by the complete data of the image to 
create a new challenge. A simple scheme is to interpret the challenge as a request to retxmi the 

20 intensity values at certain pixel locations specified by the challenge string. 

A secondary augmentor can conditionally alter the challenge posed to the client system using 
locally modifiable parameters. By being able to change the augmentation fimction as needed, the 
security of the system is greatly enhanced. One simple secondary augmentation is to add a 
25 translation to the selected locations described above. 

The server maintains the list of sensors with their secondary augmentation parameters. A 
somewhat less secure method would have the client send its augmentation parameter directly to 
the server every time. On receiving the authentication tag, the server verifies the response by 
first computing the response on the image/signal and the augmented challenge. The augmented 
30 challenge is then computed by consulting the table of the sensor and augmentor parameters. 
Then, if the response and the tag match, the image/signal is considered authentic otherwise it is 
rejected. 

An embodiment of the present invention in a networked application is shown in Figure 1. The 
local machine 100 comprises an integrated image sensor 110 and the client 120. The local 
35 machine 100 is connected to the server 140 over a network 130. Computers used as clients 120, 
networks 130, and servers 140 are well known. Sensors 110 include fingerprint scanners, video 
cameras, still cameras, microphones, iris scanners, hand geometry sensors, electronic smell 
sensors. 

The client shown in Figure 2 interfaces with the server through a known network interface 230. 
40 The client comprises one or more memories 210 and other known resources needed to execute 
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the client process 240. The sensor interface 220 handles the interactions with the integrated 
image sensor 110. The sensor interface controls acquisition and sequencing parameters of the 
sensing device over a data connection such as ISA bus, PCMCIA interface, PCI bus, USB port, 
RS-232 serial port, parallel port, IEEE 1394 serial bus. The client process 240 is described in 
5 figure 4. 

The block diagram of a hardware embodiment of the sensor and responder in a chent is shown in 
Figure 3. The sensor area 310 is the physical area for sensing the multimedia signal. The sensor 
can be a CCD light sensitive device, CMOS electrostatic imager, MEMS pressure transducer. A 
hardware integrated processor 320 acts as the responder. The integrated processor responds to 

10 challenges augmented by the challenge augmentor 330 and the sensed multimedia signal. The 
challenge augmentor can be easily replaced or reprogrammed so that the challenges can be 
augmented easily in many different ways. For instance, the augmenter might be able to compute 
either of two selectable functions, "xl+" and "xlO+". Bank A might use function "xl+" in all its 
units, while Bank B might use "xlO+" in all of its. Or, alternatively, for even number transactions 

15 function "xlO+" might be used, and for odd numbered transaction '*xl+" is used. In a preferred 
embodiment the sensor area 310 and integrated processor 320 and challenge augmenter 330 are 
fabricated on the same pieice of silicon. In another preferred embodiment these 3 pieces are 
enclosed in a tamper-resistant module that may be of a conventional design; e.g. a 
wire-embedded epoxy package in which intrusion is sensed by a change of the electrical 

20 resistance of the wire. 

Figure 4 is a flow chart of the client process 240 running on the integrated processor 320. The 
client process receives the image (or multimedia signal) in step 410. On receiving the challenge 
the augmentor in step 420 combines this with data fi-om the image, then in step 430 computes a 
valid response to the augmented challenge. The response to the challenge is outputted in step 
25 440. A typical challenge might be "3, 10, 50". This would be augmented by appending all the 
pixels values of the image (in scan order) to the end of the challenge string. The client process 
then selects the 3rd, 10th and 50th pixel value from this sequence to generate an output response 
such as "133, 92, 176". 

A preferred embodiment of a server 140 is shown in figure 5. Over the network 130, the server 
30 receives the signal and the response fi*om the client. In a table 510 or by other methods, the server 
stores the augmentation configuration for each sensor. The responder imit 520 computes the 
response to the augmented challenge and the signal. If the response comparator 530 finds the 
response received firom client and the responder unit 520 to be same, it outputs OK or accepts the 
multimedia signal. Otherwise, the response comparator returns an error or rejects the multimedia 
35 signal sent for validation. Continuing with the above example, the server knows that the 

challenge "3, 10, 50" has been issued. It also has access to the full digital image (typically sent in 
the clear). It then uses its secret knowledge of client process 240's responder program to interpret 
the challenge to mean the 3rd, 10th, and 50th pixel of the image (as opposed to, say, the 30th, 
100th, and 500th pixel). So the server also generates the response "133, 92, 176" and checks if 
40 this matches what was sent along with the image. 



YO898-0524_Nalini_3-23-99 



6 



A business process based on the invention is presented in Figure 6. The clients initiates a request 
to start a transaction by sending the sensor number to the server in step 605. For instance, a 
customer wishes to purchase a shirt at a departmental store using a credit card. The local store 
terminal reads the credit card and transmits the credit card number and the transaction amount to 
5 the transaction server. The transaction server then requests a challenge from the authentication 
server in step 610. The authentication server generates a challenge such as "3,10,50" in step 615 
and communicates to the transaction server. The transaction server in turn transmits the challenge 
to the client (i.e., the local store terminal) as shown in step 620. 

On receipt of the challenge by the client, the integrated sensor and responder is presented with 
10 the challenge in step 625. The integrated sensor senses the image (or signal) in its sensing area in 
step 630. For instance, the digital representation of the ridges and valleys of the fingerprint is 
formed. The challenge gets augmented in step 635 by the integrated processor using the 
replaceable challenge augmentor. For example, the challenge augmenter could be the function 
"xlO+" which takes the original challenge "3, 10, 50" and converts it to "30, 100, 500" and also 
15 appends all the pixels values in the acquired image to form, in effect, a new, more complicated 
challenge. The response to the augmented challenge is computed (e.g./'133, 92, 176") in step 640 
and is communicated to the client in step 645. The client (i.e., local store terminal) transmits the 
signal(i.e., fingerprint image), the response to the challenge, and the sensor identification number 
(e.g., "S/N 1234") to the transaction server in step 650. 

20 On receipt of the signal, response and id, the transaction server transmits all the parts to the 
authentication server in step 655. In step 660, the authentication server computes the response to 
the augmented challenge and compares with the response received from the client in step 665. To 
do this, it first looks up the augmentation fimction being used by the store terminal (i.e. sensor ID 
"S/N 1234" is registered as using fimction "xlO+"). It then augments the challenge it issued in 

25 the appropriate way and computes the expected response ("3, 10, 50" becomes "30, 100, 500" 
which results in "133, 92, 176"). If this matches the transmitted response, the authentication 
server knows that the associated image was taken in response to the issued challenge and hence 
is newly acquired, not a resubmitted stored image. 

Using step 670, the authentication server communicates the verdict to the transaction server. If 
30 the verdict was not to accept (step 675), the client transaction is canceled in step 680. In our 
example, the customer would be denied his new shirt. It might also be desirable to record for 
later investigation the fact that a presumably non-live image was received purportedly from that 
particular store or of that particular customer. Otherwise, in step 685 the transaction server 
transmits the image (signal) to the biometrics server. 

35 The biometrics server matches the features of the signal with the template database in step 690 
and send its verdict to the transaction server in step 692. This step typically involves the 
comparison of the signal to a stored biometrics representation. For instance, are the minutiae 
from the fingerprint image (signal) in approximately the same location as those on file for the 
customer requesting the transaction. 



YO898-0524_Nalini_3-23-99 



7 



If the verdict is to accept, the transaction server executes the transaction in step 696 and sends 
acknowledgment to cUent in step 698 and completes the transaction. Otherwise, control is 
transferred to step 680 where the transaction is canceled. 
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